Cooperative routing between traffic control device and multi-server application

ABSTRACT

A method, apparatus and programmed storage device for routing data through a communications network. More specifically, a programmable traffic manager is programmed with at least one application level directive and the data is routed through the network to one of the network nodes using the programmable traffic manager, which is programmed in accordance with the application level directive. In a particular example of this invention, a request from a client is routed by the programmable traffic manager to at least one a plurality of servers hosting an application, where the programmable traffic manager is routed in accordance with the application level directive.

BACKGROUND OF THE INVENTION

The present invention generally relates to networks, distributedapplication systems, and application-oriented networking. Morespecifically, the present invention relates to methods and apparatus forrouting requests through networks to distributed applications.

DESCRIPTION OF RELATED ART

The International Standards Organization (ISO) Open Systems Interconnect(OSI) model for computer communications comprises seven layers: (1)Physical, (2) Data Link, (3) Network, (4) Transport, (5) Session, (6)Presentation, and (7) Application. The objective of this model is toprovide for computer networks whereby end-systems (clients, hosts,nodes, servers, etc.) can be connected in order to interchangeinformation. The ubiquitous network generally available to all has cometo be known as the Internet, while more restrictive networks are knownas intranets. These types of networks typically use packet-switchingtechniques to transfer information between end-points, in contrast tocircuit-switching techniques that have been typically used for publictelephone networks.

In lieu of establishing a dedicated communication line between twocommunicators, packet-switching encourages shared communication links.In packet-switching, information to be transmitted is divided intopieces and each piece is transmitted independently though the networkordinarily comprised of a labyrinth of connections. Each piece is knownas a packet, which is the smallest unit of information that can betransmitted though the network. All the packets needed to transfer oneunified information entity do not necessarily have to take the same pathfrom source to destination, as is the case in a circuit switchednetwork.

Thus, when employing packet switching, information is disassembled at ornear the source into data packets; each data packet flows though thenetwork independently; and at or near the destination, the data packetsare re-assembled to produce the original information dispatched at thesource.

Popular implementations of the OSI Network and Transport layersrespectively are the Transmission Control Protocol (TCP) and InternetProtocol (IP), together referred to as TCP/IP. TCP controls the flow ofinformation on an end-to-end basis, for example between two end-systems.IP controls the flow of information on a hop-by-hop basis, for examplebetween two intermediaries in the sequence of nodes (or systems) visitedby a packet as it traverses the network from the originating node to thefinal destination node. The IP layer provides connectionless, unreliabledelivery of packets. The TCP layer employs the IP layer to providereliable, virtual circuits.

At the Network layer, routers are often employed for making decisions asto the path to take for movement of packets between networks. Routersmake routing decisions according to four basic algorithm types: staticrouting; isolated dynamic routing; centralized dynamic routing; anddistributed dynamic routing. In all cases, the algorithms seek tooptimize network-oriented attributes, such as network performance,network throughput, network quality of service, and so forth.

Relatively new but nevertheless known in the prior art are methods forrouting requests in a content-aware fashion. Such techniques go beyondconsideration of just network topology and network related issues(network performance, network reliability, etc.), but also take intoconsideration the payload being delivered—for example, application levelattributes.

Software applications, such as On-Demand Routers (ODRs), act asintermediaries between clients and servers providing the capability toexamine individual requests and route each according to polices inobservance of metrics. Such software routers offer substantial value forthe considered distribution of requests amongst a pool of eligibleservers to achieve desirable outcomes such as improved application-levelperformance, meeting application-level quality of service agreements,enforcing application-level security boundaries, and so forth.

The On-Demand Router is considered a proxy, a specific type ofapplication server, that routes HTTP requests to application serversthat then perform the work.” The application server is embodied insoftware. Seehttp://publib.boulder.ibm.com/infocenter/wxddoc51/index.jsp?topic=/com.ibm.wasxd.doc/todoecnfgodr.html.

A non-programmable traffic manager (like its software counterpart ODR)has built in load balancing algorithms upon which routing decisions aremade, such as:

-   -   Fastest: Passes user connection to available server that        responds the fastest;    -   Round Robin: Sends traffic to the next available server in a        predetermined sequence;    -   Least Connections: Connects the user to the server with the        least number of current connections;    -   Ratio: Performs ‘ratios’ on the server that best fits the        request, according to a system that assigns weights for various        factors, such as server capacity.

See: http://www.f5.com/f5products/products/bigip/ltm/lbl.html. Aprogrammable traffic manager allows decisions to be made based upon (forexample) a user defined algorithm. Programming of the router is left asan exercise for the user.

However, such solutions, while beneficial, can be considerably improvedupon. Software routers can be expensive to acquire and maintain, areoften complex to configure, are not standardized, and are generally slowperformers relative to firmware/hardware routers. Additionally, suchproprietary software routers make it practically impossible tosimultaneously utilize more than one vendor's server and routerplatforms.

SUMMARY OF THE INVENTION

To significant advantage, the present invention has none of the abovelimitations. The present invention facilitates employment ofprogrammable traffic management devices to produce outcomes formerlyattainable only through proprietary software solutions, such asOn-Demand Routers. By doing so, all the advantages of the prior art arerealized in addition to several new and desirable features including:improved performance; reduced cost; increased flexibility andinterchangeability; application-level middleware directed routing; andapplication-level feedback for routing adjustability.

The present invention is one way to program a programmable router(implemented in hardware or software) so that applications themselves(or some representative or delegate) control routing by means ofapplication-oriented directives. The present invention is routerindifferent with respect to implementation in software or hardware orfirmware or some combination thereof. Any programmable router willsuffice. What's important is that the router implementation expose aprogrammable interface so that incoming requests are routed according touser/application supplied algorithms that consider application-orienteddirectives.

One aspect of the present invention is to supply traffic managementalgorithms that consider application-oriented directives. With thisinvention the back-end clusters feed “executable code” into the router,instead of feeding runtime state information into a pre-defined routingalgorithm. This invention provides for injecting the algorithm itselfinto the router. By having the middleware/application so inject thealgorithm, the router can more easily perform application-orientedrouting tasks.

The teachings of the present invention will become apparent from thefollowing detailed description of illustrative embodiments thereof,which is to be read in connection with the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram showing a runtime architecture for routing ofrequests to distributed applications by means of a traffic managerprogrammed according to an application's desires.

FIG. 2 is a diagram showing a method for programming a traffic manager.

FIG. 3 is a diagram showing a method for traffic manager request routingaccording to externally applied configuration criteria.

DETAILED DESCRIPTION

The present invention will be described below using an illustrativeexample centered on an Internet based client-server model. However, itis to be understood that the present invention is not to be limited assuch and is applicable to any request-based environment wherebyapplication-oriented routing capabilities are desirable.

An “application-oriented or application level directive” may be anautonomic reflex that an application produces in response to somestimulus. For example, if an application determines that its quality ofservice criteria are not being met on node A but are far exceedingrequirements on node B, it may produce an “application-orienteddirective” to route selected application destined traffic away from nodeA and toward node B.

Referring initially to FIG. 1, an architecture is illustrated accordingto an embodiment of the present invention. A plurality of requests 110are submitted by clients for processing. Said requests may originatefrom a diverse set of clients (personal computers, personal digitalassistants, cellular telephones, etc.), and may find their way to a datacenter for processing via a multitude of paths (wireless and/or wiredcommunications, Intranet, Internet, WAN, LAN, etc.). The presentinvention is not limited to any particular communications network.

A client request 110 may also originate from a node hosting anapplication server. An application server 150-152 may be comprised of ageneral purpose server hosting one or more applications. A node 140-142may host one or more application servers 150-152. A request may or maynot result in information returned to the requester. The presentinvention places no requirements on the type of a request or the type ofnode 140-142 or application server 150-152 that processes a request,though for purposes of clarity an example below describes one manner inwhich the present invention can be applied for routing of clientinitiated HTTP requests to applications on servers that handle suchrequests.

At some boundary, subsequent to any outlying routing, a client request110 arrives at a Traffic Manager 120. The Traffic Manager 120 isresponsible for routing each arriving request to one of a plurality ofapplication instances 150-152 deployed on corresponding nodes 140-142within a cluster 130. According to an embodiment of the presentinvention, the Traffic Manager 120 routes each client request 110 to thedesired application instance 150-152 by consulting programmed rules.Programmed rules are executable polices (directives) specifiable byapplications themselves and/or by other suitable providers.

It is to be understood that the present invention is not to be limitedto any particular configuration, such as that illustrated by FIG. 1,which is provided for illustrative purposes. For example, a plurality ofTraffic Managers 120 or a plurality of clusters 130 may be may beutilized. One skilled in the related art can easily contemplate theseand other suitable configurations.

The programmable Traffic Manager 120 is endowed with severalcapabilities. Through a published Application Program Interface (API),it is able to receive and store rules that govern how it should routerequests. In addition, during processing of requests the Traffic Manager120 is able to fetch, interpret, and apply these rules that instruct theTraffic Manager 120 on the specific content of arriving requests toexamine for use in determination of where to route each request. Basedupon the programmed rules and the content of each request, the TrafficManager 120 routes each request accordingly.

Without programmed rules, the Traffic Manager 120 has no intelligence.That is, the Traffic Manager 120 does not know how to examine or routerequests until it is provided one or more rules that direct itsbehavior. Unlike the prior art, whereby routing at the TCP/IP Networkand Transport layers failed to consider application-level routing, thepresent invention facilitates programmed rules for directing networktraffic in correspondence with the needs and desires of the targetapplication. That is, the present invention provides application-level(or application-oriented) routing through rules-based programming of thetraffic manager.

For example, initially Application A may desire that requests to tradeIBM stock be directed to the Application A instance on Node 1. Later,Application A may provide new rules so that newly arriving requests totrade IBM stock be directed to the Application A instance on Node 2. The“needs and desires of the target application” are embodied in thecurrently active application provided set of algorithms (or “rules”).The application provided algorithms can be as simple or as complex asthe programmable router is capable of accepting though its API. Theapplication provided algorithms can examine the content of requests (forconsideration in where the request should be routed) arriving at theprogrammable router to the degree permitted by the it. The applicationitself does not have to directly provide the algorithms—instead a proxyor representative may do so.

The application-level rules programmed into the Traffic Manager 120 maybe static or dynamic in nature. The application-level programmed rulesmay be set once then remain unchanged for long periods of time, or theymay change frequently. Control of routing changes may come directly froman application, or from an application monitor, or from an applicationanalyzer, or from an application proxy and/or other entity. It is to beunderstood that the present invention is not to be limited to anyparticular application-level routing control authority.

As a specific example that will be used throughout, suppose that thetarget application is a stock quote server, and that a client is onethat is able to request stock quotes based on user entered tickersymbol. A client may issue an HTTP request 110 to obtain a stock quotefor ticker symbol IBM. Further, suppose that any of the Application Ainstances 150 can satisfy the IBM stock quote request. Under somecircumstances, it may be desirable to “round robin” each clientinitiated IBM stock quote request 110 amongst the plurality ofApplication A instances 150 able to service them. But under othercircumstances, it may be desirable to route all such requests 110 toexactly one Application A instance, say the one running on Node 1 (140).

The Programmable Traffic Manager 120 is able to interrogate each requestto obtain one or more attributes, for example stock ticker symbolassociated with a request, and apply the programmed rules accordingly todetermine destination node. Programmed rules may be changed dynamicallyto effect revised routing decisions, for example re-routing requestspreviously destined for Node 1 (140) to Node 2 (141).

In one embodiment of the present invention for partitioning of requests110 amongst application instances 150-152, facilities are provided forapplications to indicate desired routing by means of one or more RegularExpressions (REs), which are well known in the related art. Theapplication may, for example, specify a RE such that all requests forstock ticker symbol IBM are routed to application instance A (150)running on Node 1 (140). Application-oriented routing information, be itstatic or dynamic, one or multi-dimensional, is specifiable throughExtensible Mark-up Language (XML) data and/or API calls. These datacomprise the Metrics and Policies 160.

The present invention places no restriction on the types of static anddynamic application-oriented data that can be considered for Metrics andPolicies 160. Such data need not be restricted to applications, per se.Metrics may include disk space available, CPU utilization, memoryutilization, priority, and myriad other dimensions at the applicationlevel, system level, or at some other abstraction level. Policies may bespecified as REs or in any other suitable form. Policies may target oneor more applications. An individual Policy may span multipleapplications.

One important aspect of the present invention is the ability to deploysuitably formed executable policies into the router (Traffic Manager120). This novel aspect greatly improves upon the state of the art, forexample where a framework simply updates state variables for some wellknow policy. Since the deployed artifacts are in fact executable, thepresent invention provides for significantly advanced flexibility andexpression in runtime routing decision making.

The Routing Scheduler 170 provides a means for Application Policies andMetrics 160 to be deployed to one or more Traffic Managers 120, afterhaving been transformed into a suitable form by a Rule Manager 180.

The Routing Scheduler 170 may receive application-oriented feedback overtime that effects routing. Continuing the above example, suppose thatoriginally stock ticker symbols have been partitioned so that IBM stockquote requests 110 are directed by one or more Traffic Managers 120 toApplication A 150 on Node 1 (140) for service. At some point theapplication decides to move the IBM partition to Node 2 (141), perhapsattempting to achieve improved response time. This revised ApplicationMetric and Policy 160 information is provided to the Routing Scheduler170, which, in turn, employs the Rule Manager 180 to transform anddeploy correspondingly updated rules to one or more Traffic Managers120. Subsequently, IBM stock quote requests 110 arriving at saidre-programmed Traffic Managers 120 are directed to Node 2 (141) forservice.

In one specific implementation, an F5 BIG-IP Application ManagementTraffic Device was used as the Traffic Manager 120. A single Metric 160comprised of a string indicating partition name IBM, and a single Policy160 to extract stock ticker symbol from each request 110 were employed.A Routing Scheduler 170 employed a Rule Manager 180 to formulateprogrammed rules deployed to the Traffic Manager 120 so that requestsfor stock quotes for ticker symbol IBM were routed to Node 1 (140).

It is to be understood that the present invention is not to be limitedto any particular traffic manager implementation. It is to be furtherunderstood that the present invention is not to be limited by use ofprogrammed rules for controlling traffic manager behavior. One skilledin the art can readily employ other suitable variability mechanisms suchas databases or ontologies to replace or supplement programmed rules.

The present invention distinguishes itself from both quality of servicenetwork routing and TCP/IP distributed application network routingthough its application-oriented routing capabilities. Neither of thesepreviously existing techniques offer the types of controls necessary forprecision application directed routing.

Referring now to FIG. 2, a method for programming a traffic manager isillustrated. A plurality of metrics and policies 210-250 from one ormore application-oriented sources in one or more formats are deliveredto a Routing Scheduler 170. Such sources may include, for example, XMLdata attached to the application that specify application policies andXML data attached to the system that describe system attributes. Thisapplication-oriented information may be pushed to or pulled by theRouting Scheduler. The application-oriented metrics and policiesinformation may be assembled once per deployment or may be ongoing. TheRouting Scheduler compiles the application-oriented metrics and policesto arrive at an estimated application-oriented desired routing. In thecase of ongoing dynamically adjustable routing, as more informationarrives it is compiled to produce new application-oriented desiredrouting estimates.

Application-oriented metrics and policies comprise application-orienteddirectives. Such directives may include application-level metrics andpolicies, for example definitions for stock ticker symbols such as“IBM”, “GE”, and “EBAY” and corresponding desired trade response times;system-level metrics and policies, for example application CPUconsumption limits at each node; and enterprise-level metrics andpolicies, for example the relative shares of memory allocated to eachapplication enterprise-wide.

An application oriented metric may be a metric of interest or use to anapplication more so than the system upon which the application runs. Forexample, a stock ticker symbol may be of use to one application butwould likely not be of interest or use the system and otherapplications. A non-application-oriented metric may be a system metric,such as CPU utilization.

An application oriented routing policy is a routing policy influenced ordirected by an application, not just by the system. For example, asystem load balancing policy may route each newly arriving request tothe node with the lowest CPU utilization, regardless of the needs anddesires of the application running on those nodes. Whereas, anapplication-oriented routing policy may route requests for IBM stocktrades to Node 1 and requests for GE stock trades to Node 2.

In the former case, the application may not be able to use aggressivecaching techniques, since each request to trade IBM stock may be sent toa different node. In the latter case, the application might employaggressive caching techniques with the knowledge that all requests totrade IBM stock will arrive at Node 1.

Below is an XML representation of application-oriented policies thatroute user requests for stock trades. Request for IBM stock trades wouldbe routed to Node 1, while request for GE stock trades would be routedto Node 2, etc. <Policy StockTicker=IBM Node=1 /> <Policy StockTicker=GENode=2 /> <Policy StockTicker=EBAY Node=3 /> <Policy StockTicker=GOOGNode=3 />

Another example of a policy, in this case a dynamic policy, for routinguser requests to trade IBM stock is shown below in XML format In thiscase, the router will attempt to route only to nodes where the responsetime is less than 0.5 sec.

<Policy StockTicker=IBM ResponseTime<0.5 sec />

The Routing Scheduler 170 may be implemented as a Java program thatanalyzes application provided metrics and policies to produce one ormore strategies for routing of requests. For example, the analyzer maydetermine that a very large number of requests to trade IBM and eBaystocks are arriving, while a relatively small number of requests totrade GE and McDonalds also arriving. Further, the application may havea high priority policy of minimized queue delay. The analyzer maydetermine that the best strategy is to route requests for IBM and GEstock trading to Application A on Node 1, and requests for eBay andGoogle stock trading to Application A on Node 2.

The Rule Manager 180 may be implemented as a Java program thattransforms strategies produced by the Routing Scheduler 170 into a formexecutable by a programmable router (i.e.—Traffic Manager 120) accordingto its API. For example, in one implementation the Routing Scheduler 170produces one or more Regular Expressions (REs), well known in therelated art, as the representation of routing rules to be carried out bythe router. The Rule Manager 180 takes said one or more REs and does atransformation (if necessary) into a form recognized by the TrafficManager 120, then invokes the corresponding Traffic Manager 120 API toadd and/or modify and/or remove the routing rules executed by it atruntime for routing of arriving application service requests.

In one embodiment, the Routing Scheduler 170 is a Java program thatsimply takes application policies specified as REs in XML format andpasses them onto the Rule Manager 180 as is. In another embodiment, aJava program retrieves application policies specified in an applicationspecific format thorough an application specified API, then employs oneor more plugin components to transform the representation of policiesand metrics into a common form, such as REs prior to passing them ontothe Rule Manager 180.

Once an application-oriented desirable estimate has been achieved, theRouting Scheduler 170 employs a Transformer 180 to produce deployablerules. The Routing Scheduler may also deliver, via subscription oron-demand request, the application-oriented desired routing estimate toother interested parties, such as an Application Manager 265, directlyor though some intermediary. Once transformed, the rules can be deployedto configure a Traffic Manager 120.

An “application oriented desired routing estimate” is the routingdesired by the application, which may not be doable in reality. Forexample, an “estimate” may be to route requests for IBM stock trades toNode 1, and requests for GE stock trades to Node 2. However, when arequest arrives, Node 1 may have since gone down. In that case, therequest may be routed to another node for service, not the applicationdesired one.

The Traffic Manager 120 expects the routing specification in a certainformat in order to be programmed according to its API; and theapplication may specify is policies in a format that does not preciselymatch the format understood by the Traffic Manager; and thus the RuleManager 180 does a transformation between these formats. Then theTraffic Manager 120 receives and implements the application-orientedrouting specifications deployed by the Rule Manager.

For example, one particular format specifiable by applications isRegular Expressions in XML, which are transformed into a format suitablefor use by F5's Traffic Manager and are deployed by invoking the TrafficManager's API to load the transformed REs.

Referring now to FIG. 3, a method for traffic manager request routingaccording to externally applied configuration criteria is illustrated. Atraffic manager receives a request (310) and routes it to theappropriate application instance in correspondence with a generatedrules configuration (320) produced by the Rule Manager 180.

Externally applied configuration criteria may include algorithms (or“rules”) that control how the routing of requests is to occur, areconstructed outside of the router proper, and then fed into the routerby means of the router's API.

The output of the Rule Manager 180 is delivered to the Traffic Manager120 according to the latter's API. The Traffic Manager then utilizes theadditions/changes/deletions so delivered for all applicable routing ofrequests from that point forward.

While the foregoing is directed to the illustrative embodiment of thepresent invention, other and further embodiments of the invention may bedevised without departing from the basic scope thereof.

1. In a communications network having a plurality of nodes, a method ofrouting data through said network, said method comprising: programming aprogrammable traffic manager using at least one application leveldirective; and routing said data through said network using saidprogrammable traffic manager, wherein said data is routed in accordancewith said application level directive, and wherein said programmabletraffic manager causes said data to be routed to one of said nodes.
 2. Amethod as recited in claim 1, wherein said nodes are servers hosting anapplication and said data is a request from a client.
 3. A method asrecited in claim 1, wherein said traffic manager is programmeddynamically by said application.
 4. A method as recited in claim 1,wherein said application level directive is established by at least oneof the following: an application, an application monitor, an applicationproxy, and an application analyzer.
 5. A method as recited in claim 1,wherein said application level directive uses at least one of thefollowing: application-level metrics, system-level metrics, andenterprise-level metrics.
 6. A method as recited in claim 1, whereinsaid application level directive comprises at least one of thefollowing: application-level policies, system-level policies, andenterprise-level policies.
 7. A method as recited in claim 1, whereinsaid application level directive interrogates said data for at least oneattribute to determine said one node to which said data is to be routed.8. A method as recited in claim 1, wherein said programmable trafficmanager is dynamically re-programmed through an application programinterface for dynamic re-configuration of said application leveldirective.
 9. A method as recited in claim 8, wherein said reprogrammingresults in a routing destination change for said data.
 10. A method asrecited in claim 1, wherein at least one application level directivecomprises policies and metrics specified as at least one regularexpression
 11. An application-oriented routing method for routingrequests to a plurality of servers using a programmable traffic manager,said method comprising the steps of: employing at least one applicationlevel metric and at least one application-oriented policy to estimate atleast one request route; transforming said estimated route into a formsuitable for said programmable traffic manager; deploying saidtransformed estimated route onto said programmable traffic manager; androuting at least one request according to said deployed estimate routethrough said programmable traffic manager.
 12. A program storage devicereadable by a digital processing apparatus and having a program ofinstructions which are tangibly embodied on the storage device and whichare executable by the processing apparatus to perform a method ofrouting data through a communications network, said method comprising:programming a programmable traffic manager using at least oneapplication level directive; and routing said data through said networkusing said programmable traffic manager, wherein said data is routed inaccordance with said application level directive, and wherein saidprogrammable traffic manager causes said data to be routed to one ofsaid nodes.
 13. An apparatus for routing data over in a communicationsnetwork having a plurality of nodes, said system comprising: aprogrammable traffic manager; and a processor for programming saidtraffic manager with at least one application level directive, whereinsaid programmed traffic manager causes said data to be routed to one ofsaid nodes.